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STATUS OF CLAIMS 

This application contains claims 1-27 and 58 - 67, ali of which stand rejected as 
obvious over prior art Claims 28 - 57 were canceled. M rejections of ciaims 1-27 and 58 ~ 67 
are appeased. 
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GROUNDS QF REJECTION TQ BE REVIEWED QFi APPEAL 

1, Whether the outstanding §103 rejections to claims 1-27 and 66 over Shneiderman et 
at, "Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos," July 2000 
("Shneiderman") in view of EP. 0 678 816 A2 to Mizoquchi (" Mizoguchn should be 
reversed. 

2, Whether the outstanding §103 rejections to claims 58-65 over Shneiderman in view of 
U.S. P. 5,485,611 to Astle ("Astie") should be reversed. 

3, Whether the outstanding §103 rejections to ciaim 67 over Mizoguchi in view of 
Shneiderman should be reversed. 
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ARGUMENT 

The obviousness rejections to claims 1-27 and 58-67 must be reversed because the 
references, even when taken collectively, fail to teach or suggest the subject matter of the 
pending claims. The arguments presented in the Examiner's Answer fail to remedy the defects 
of the cited art identified by Appellants in the Appeal Brief and previous Responses. 

The Examiner continues to assert that Shneiderman alone or in combination with other 
references renders the claimed invention obvious. However, the Examiner's argument reiies on 
features not present in the cited art. In many cases, the Examiner misunderstands the cited 
art, the recited claims, or both. The art cited by the Examiner simply faiis to make out a prima 
facie case of obviousness of the recited claims. 

NONE OF THE CITED ART TEACHES THE USE OF SOCIAL GROUPS OR GROUPS OF 
USERS OF A MEDIA ARCHIVAL SYSTEM 

The Examiner's rejections give short shrift to the claim elements that use data 
representing social groups to index digital media items. Essentially, the Examiner beiieves that, 
if Shneiderman 's system can record information relating to people or events generally, then it 
must record data of social groups. In this regard, the Examiner's analysis ignores significant 
elements from the claims. The ciaims do not refer to data of groups generally; instead, the 
claims require operators to identify themseives as members of a social group and, thereafter, 
presents candidate indexing information to the operator when digital media items are being 
indexed. In this regard, the ciaims define a system that distinguishes over the prior art. 

Claims 66 and 67 are exemplary. They recite, in relevant part: 

receiving a user input identifying a social group to which an archiving 
user belongs, 
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building a database that includes •♦• index information for the digital media 
items, each instance of index information created from archiving input data 
identifying a user's response to a query that identifies a plurality of event 
types previously registered as associated with the social group, and 
persons previously registered as members of the social group. 

Under the Examiner's analysis, Shneiderman 's system records information relating to events 
and people and, therefore, he must disclose queries that identify both event types and persons 
previously registered with a social group. This simply is untrue. Shneiderman 's system records 
this data in the abstract. There is no hint or suggestion anywhere in the reference that any 
such data is related to a data item that represents a social group. 

The Examiner's rejections also rely on Mizoquchi and Astle to supplement the prior art 
rejections but these references also fail to disclose use of social group data. The cited 
disclosure from Mizoquchi , for example, merely shows that when a photograph is being 
displayed, one may press keys representing persons and piaces to recall previously stored 
information regarding such events. There is no reference to social groups whatsoever, 
Similarly, Astle shows only that a video segment may be indexed based on a timestamp or a 
representative frame from the video; there is no mention of sociai groups. 

Thus, even if the disclosures of these references were considered together, this body of 
art does not teach or suggest the advantages of episodic memory systems that are claimed 
here. There is no reference to authenticating users as members of social groups, nor is there 
any disclosure to demonstrate archiving of media items based on their relevance to person data 
or event data values that are registered with the social group. In this regard, the Examiner 
disregarded claim language and important teachings from the specification. 

Applicant's Appeal Brief challenged the Examiner's rejections on exactly this point. In 
his Answer, the Examiner provided no explanation to demonstrate how the prior art allegedly 
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shows this subject matter. He merely asserted without explanation that this subject matter is 
present One would expect that, had Shneiderman really indexed digital media items using 
event or person values of sociai groups, that the Examiner could have given a more 
comprehensive expianation of exactiy how Shneiderman discloses this subject matter. In point 
of fact, he does not; his system does not use any such information in this manner and therefore 
the rejections must be reversed. 

Below, Applicants identify analytical errors in the Examiner's Answer on a ciaim-by-claim 

basis. 

Independent Oa i ms 58 and 62 

Claims 58 and 62 recite, in part; 

authenticating an operator as a member of a group of users, 

identifying candidate identification values based upon the group with whom 

the operator is authenticated, 

querying the operator for selection of identification data to be associated with a 
digital media item, the query identifying the candidate identification values and 
including valid selections of an event type and persons from the group 

and time. 

Again, although the ciatm requires that candidate identification values be based on a group of 
users with whom the operator has been authenticated, none of the cited art teaches or 
suggests this concept. The Examiner's analysis merely indicates that because the system in 
Shneiderman might be used by multiple operators, Shneiderman must disclose these elements. 
See Examiner's Answer, p, 11. Such a speculative assertion cannot uphoid an obviousness 
rejection. While multiple operators might use the system described in Shneider man to annotate 
photographs, there is no indication that the software authenticates an operator as a 
member of a group of users, or that the software operates any differently based on the 
person using it. In fact, SMeiderman suggests that his software is incapable of doing so since, 
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if multi-user functionaiity is added, the software will have to provide for "appropriate resolution 
of conflicts." Shneiderman, p. 6, lines 8-12 (citations omitted). That is, Shneiderman does not 
disclose that his system can arrange users into groups, or discern if a particular user is a 
member of a group. The reference is entirely silent on this issue. 

The Examiner further argues that Shneiderman's system discloses the claimed step of 
"identifying vaiid seiections of an event type and persons from the group" because the 
system prompts the user to enter retrieval criteria including names, event, time and keywords 
which are annotated by the user. Examiner's Answer, p. 11. Again, this simply is incorrect. 
Shneideiman's system merely searches previous annotations. There is no suggestion that an 
operator is presented with valid selections of an event type and persons from the group 
with whom the operator is authenticated. As previously explained, Shneiderman simply 
does not describe or suggest arranging users of the system into groups or authenticating an 
operator as a member of a group. 

Dependent Claims 59 and 63 

Claims 59 and 63 recite that "candidate identification vaiues for persons include names 
of group members." In his Answer, the Examiner argues that Shnejderman's. Fig. 4 depicts 
an "operator" and that the names available for annotation are a "group" as recited in the 
claims. This is not only incorrect, it is a blatant misreading of the claim. The "groups" 
identified by the Examiner, i.e., the group of people in the photograph and the list of names 
available for annotation, have no relation to the group of users recited in the claims. 
Furthermore, the master list in Shneiderman is unrelated to the operator of the system - it 
merely presents a list of possible annotations, based on names previously used to annotate 
photographs. These names have no relationship to groups of users of the annotating system. 
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Contrary to the Examiner's assertions, the fact that a set of people is pictured in a photograph 
or specified in a list of names does not imply that those same people form a group of users of a 
system for annotating the photograph. 

Dependent Claims 60 and 64 

Claims 60 and 64 recite in pertinent part "a flag that distinguishes high point items from 
other items." As clearly explained in the specification, a "high point," allows a user of the 
system to identify an item as associated with a particularly memorable event. See specification, 
p. 4, lines 4-8; p, 12, lines 25-31. The Examiner asserts that any of the attributes in 
Shneiderman, may be a "memorable indicator," and argues without explanation that annotation 
data such as "July 5, 2006, 12:30 p.m." (i.e., date and time) represents a particularly 
memorable event. Examiner's Answer, p. 12, Shneiderman's disclosure suggests, however, 
that such data is solicited for every stored photograph, which belies the Examiner's conclusion. 
The free-text annotation fields in Shneiderman allow any information to be entered, and do not 
include a mechanism for distinguishing, for example, a "particularly memorable event" from a 
iess-memorabie, but more thoroughly annotated event. 

Dependent Claims 61 and 65 

Claims 61 and 65 recite the use of a "trail" in archiving and retrieving media items. The 
term "trail" is clearly defined in the specification as "a logical or meaningful route through an 
archive of digital media items as perceived by the user." Specification, p. 4, lines 15-17. 

The Examiner argues that the photographs displayed in Shneiderman's Figure 4 are a 
"trail" because 'the teaching or explanation of the term trail does not importf] from the 
specification." Examiner's Answer, p. 13-14. This is the first time the Examiner has given any 



S'27287J.OOC 



-8- 



13642,'! 



Reply Brief 
Appin. No. 09/670,635 
Atty. Dkt. No. 13642/1 

indication of the basis for rejecting these claims. Previous Office Actions rejected these claims 
on the same grounds as claims 60 and 64. See Appeal Brief, p. 21; Amended Appeal Brief, p. 
22-23. 

As the Examiner should he well aware, an applicant is entitled to be his own 
lexicographer. See MPEP, § 2111,01 (III). Furthermore, "jwjhere an explicit definition is 
provided by the applicant for a term, that definition will control interpretation of the term as it is 
used in the claim." Id. (citing Toro Co. v. White Consolidated Industries Inc., 199 F.3d 1295, 
1301, 53 USPQ2d 1065, 1069 (Fed. Cir. 1999)). The specification also gives specific examples 
of the meaning and usage of the term at page 4, lines 17-31, further defining the use of the 
term in the claims. The clear definition and usage in the application should be applied in 
interpreting the claims, not a definition created by the Examiner. See Phillips v, AVVH Corp,, 
415 F.3d 1303, 75 USPQ2d 1321 (Fed. Cir. 2005) (en banc); Vitronics Corp. v. Conceptrontc 
Inc., 90 F.3d 1576, 1583, 39 USPQ2d 1573, 1577 (Fed. Cir. 1996). Interpreting the claim 
language in light of the specification, it is clear that Shneiderman has no bearing on the claims. 
Nowhere does Shneiderman teach or suggest that the photographs shown in Figure 4 are 
arranged in a "logical or meaningful route. .. as perceived by the user." The Examiner has failed 
to make out a prima facie case of obviousness, 

The Examiner's argument also fails to address each element of the claims. Specifically, 
Shneiderman does not teach or suggest "an identifier representing the media item's display 
position in a sequence of stored media items" as recited in claim 61 or ^presenting stored 
media items in a sequence as identified in the index information" as recited in claim 65. 
The figure cited by the Examiner in rejecting these claims merely shows several stored 
photographs that are available for annotation. There is no indication that these photographs 
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are associated with index information that identifies each photograph's position in a sequence. 
Again, the Examiner has ignored the language of the claims. 

Inde i |^ndent ii Ciaim iii l 

Clasm 1 recites, in part, "'receiving a user input identifying a group of users to which 
an archiving user belongs." In contrast, Shneiderman is silent as to any identifying 
information provided by a user of the system. The only user input in Shneiderman is 
annotations or search criteria, neither of which relate to an archiving user. 

The Examiner argues that Shneiderman teaches this element because a user iogs on to 
Shneiderman's system to annotate photographs. Examiner's Answer, p. 3, Appellants have 
previously explained that Shneiderman discloses no such login process. In response, the 
Examiner now asserts that claim I does not require a login process. Examiner's Answer, p. 15. 
Although this new argument is quite late, it also misses the point. The ciaim requires not 
merely a login but rather "a user input that identifies] a group of users to which [the] archiving 
user belongs," Shneiderman discloses no such thing. 

Claim 2 recites input data that "comprises a user input from another user identifying a 
group to which the other user belongs." The Examiner argues that Shneiderman allows a 
photo to contain references to multiple persons and the same person can appear in multiple 
photos. Examiner's Answer, p, 15-16. According to the Examiner, a person common to two 
photographs "wil! be the bridge between [two] photos and group of member of the photo." 
Examiner's Answer, p. 16. This indexing method is unrelated to the ciaim. The one-to-many 
relationships described in Shneiderman only relate to storing photograph annotations. There is 



S'27287J.OOC 



- 10- 



13642,'! 



Reply Brief 
Appin. No. 09/670,635 
Atty. Dkt. No. 13642/1 

oo reference to a user input identifying a group to which another user belongs. The 
Examiner's argument again confuses persons pictured in annotated photographs with users of 
an archival system. 

Claim 3 recites "defining group event types that are appropriate for members of the 
group to distinguish episodic events memorable to the group," The Examiner argues that in 
each photo in Shneiderman tt is a group event because it presents an event with a photo having 
a group of people." Examiner's Answer, p. 16. Clearly, the photographs in Shneiderman may 
display multiple people. However, the Examiner fails to explain why this renders the claim 
obvious. There is no relationship between a photo of a group of people and group event types 
specific to groups of users of an archiving system. The photographs in Shneiderman are 
unrelated to group event types, and the concept of event types associated with a group of 
users is entirely absent from the reference. 

Claims 5 and 6 are directed to the use of "high points" in archiving digital media items. 
The Examiner argues that the language of the claim is unclear and therefore any indexing 
information renders the claim obvious. Examiner's Answer, p. 17. However, the term "high 
point" is clearly described in the specification. Designating a media item as comprising a "high 
point" allows a user to associate the media item with a "particularly memorable event." 
Specification, pA An example of such an event is a group's last day of school. Other examples 
and uses of "high points" are given in the specification. See specification, p. 12. 

The Examiner has ignored the use of this term in the specification, and continued to 
apply his idea of the "piain meaning" of the phrase. This is not the correct standard. The 
meaning of a claim term should be determined according to the usage of the term in context in 
the specification. See MPEP § 2111.01 (III); Phillips v, AWH Corp., 415 F.3d 1303, 75 USPQ2d 
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1321 (Fed, Cir. 2005) (en banc); Vitronics Corp. v, Conceptronic Inc., 90 F.3d 1576, 1583, 
39USPQ2d 1573, 1577 (Fed. Cir. 1996). Based on the usage of "high point" in the 
specification, the Examiner is incorrect that any of "people, event type, iocation and date" may 
be a "memorable event" Shneiderman simply does not suggest the use of a high point as 
recited in the claims. 

Claims 7 and 8 are directed to indexing and retrieving media items based on the "media 
type" of the items. The Examiner argues that "the digital photograph is a type of media that 
can be retrieved" in Shneiderman . Examiner's Answer, p. 17 (citing Shneiderman Figure 4 and 
p. 8, lines 37-40). While a digital photograph is certainly a type of media, it has no bearing on 
whether Shneiderman teaches archiving or retrieving media items based on a media type. In 
fact, if Shneiderman organized media items by type, it would be equivalent to no organization 
at all - every media item in Shneiderman is a "digital photograph," There is no other type of 
media item described or suggested in Shneiderman. Other media types, such as an audio clip, 
would be unusable in the system of Shneiderman- there would be no way to "drag and drop" 
annotations onto the media item. The cited reference therefore does not teach or suggest the 
elements of the claims. 

Claim 9, from which claims 10-12 depend directly or indirectly, recites "receiving 
archiving input identifying a plurality of digital media items and an input identifying the 
digital media items to be associated as perceived by the user." Shneiderman only describes 

annotating a single photograph or searching based on pre-defined data fields; there Is no 
suggestion that a plurality of media items may be grouped based on an association 
perceived by the user. The Examiner again argues that these claims are rendered obvious 
by the disclosure of Shneiderman that a photograph may be associated with people and dates, 
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and that a set of photographs may be searched by person. Examiner's Answer, p. 18. This is 
incorrect. The recited claims aiiow for episodic storage and retrieval of media items, whereas 
Shneiderman only allows for annotation and retrieval based on specific pre-defined data. The 
rejection is unsupported by the cited art. 

Claim 13 recites 

receiving a user request for automatic nostalgic retrieval, 
automatically generating an initiai set of said selections, 

using the selections to retrieve and output digital media items, 
automatically modifying one or more of the selections, 
using the modified selections to retrieve and output digital media items and 
repeating the modifying, and retrieval and output steps. 

In contrast, Shneiderman requires a user to search stored photographs by a person's 
name. There is no suggestion that Shneiderman's system automatically generates or modifies 
selections. The Examiner argues that Shneiderman renders the claim obvious because both 
systems require initial user input when searching archived items. This argument ignores the 
clear language of the claim, Shneiderman's disclosure of a simple search-by-name function 
does not render the complex, automated nostalgic retrieval process recited in claim 13 obvious. 

Claims 14-27 

Independent claims 14, 15, and 17 are apparatus arid method claims with limitations 
similar to those in ciaim 1. Claims 16 and 18-27 depend from claims 14, 15 and 17. The 
Examiner presents no new arguments in support of his rejection of these claims, but rejects 
them on the same rationale as claim 1, See Examiner's Answer, p. 3, 7, 19. As previously 
explained, Shneiderman does not suggest the use of groups as recited in the claims, such as 
identifying "a group to which the user belongs" (claims 14, 15 and 17) or identifying "a digital 
media item to be archived for the group" (claim 17), As described m the Appeal Brief, the 
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dependent claims also recite additional features not found in the cited art; these features were 
not directly addressed in the Examiner's Answer. 



Independent Claims 66-67 

As noted above, claims 66-67 both recite: 



receiving a user input identifying a social group to which an archiving user 
belongs; 

building a database that includes ♦*• index information for the digital media 
items, each instance of index information created from archiving input data 
identifying a user 's response to a query that identifies a plurality of event 
types previously registered as associated with the social group, and 
persons previously registered as members of the social group 

In the Examiner's analysis, Mjzogychi and Shneiderman describe associating people's names 

with a photograph, and therefore must aiso disclose event types and persons registered with a 
social group. This is incorrect. The only "groups" of people described in Shneiderman are 
those appearing In the photographs; there is no teaching or suggestion to search stored media 
items based on a soda! group identified by a user, Mizpguchi fails to remedy this defect. 
Mizoquchi only describes searching photographs stored in a digital camera by a person, place, 
or "other data" that was previously associated with the photograph. See Examiner's Answer, 
p.8; Mizoquchi col, 10, lines 19-29, There is simply no teaching or suggestion that the 
photographs may be searched by vaiid selections of event type, persons, or time associated 
with a social group identified by an operator. The Examiner's analysis ignores the language of 
the claims. 

THERE IS MQ MOTIVATION TQ COMBINE THE CITED REFERENCES 

The Examiner argues that one of skill in the art would combine Shneiderman and 
Mizoquchi because both references "disclose the method of indexing the photo using the 
person, date time, event type as the attributes for indexing." Examiner's Answer, p. 14. As 
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previously described, the system of Mizoguchi requires a scheduling/calendaring system to 
organize photographs, white the system of Shneiderman allows for freetext input. See 
Mizoguchi , coi. 5, lines 1-7; col. 8, lines 15-47; Shneiderman , p. 4, As stated by the Examiner, 
both references allow for annotation of photographs using the same types of data. One of skill 
in the art therefore would have no reason to combine the schedule-based system of Mizgguchi 
with the freetext annotation system of Shneiderman, since doing so wouid not provide any 
apparent benefit. 

There is also no motivation to combine the photograph annotation system of 
Shneiderman with the video frame indexing system of Astje. The Examiner argues that: 

An indexing system is inherently used in Shneiderman; however, Shneiderman is 
stient in the use of indexing the photo with these attributes. On the other hand, 
Astie clearly discloses the method and process of indexing the frame. Therefore, 
the motivation to combine Shneiderman and Astle is to index with multiple 
attributes for better chance of retrieval of photo. 

Examiner's Answer, p. 19. Astie allows a video clip to be indexed based on a representative 

frame. See Astle, coi. 2 line 56 ~ coi. 3, line 6; coi. 6, lines 4-28. Such a system wouid be 

useless for indexing photographs as suggested by the Examiner, since each photograph is 

already a single frame. The video indexing system of Astle is unrelated to the photograph 

annotation system of Shneiderman, and one of skill in the art wouid have no motivation to 

combine the cited references. 
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CONCLUSION 

Again, Applicants respectfully request reversal of the obviousness rejection to claims 1- 
27 and 58-67, The Examiner's rejections fail to demonstrate that any of the prior art references 
teach or suggest the principles on which the present invention is based - archiving digital media 
items using episodic memory of a social group of users. Although the Examiner's Answer 
asserts that the prior art demonstrates this feature, he has provided no analysis to explain how 
the references use candidate information values that are based on a social group. In short, he 
cannot - none of the prior art discloses this subject matter. 

For the foregoing reasons, Applicants respectfully requests this Board to reverse the 
Examiner's outstanding rejections. These claims are allowable over the cited art. 

The Commissioner is authorized to charge any fees or credit any overpayments to 
Deposit Account No, i 1-0600. 

Respectfully submitted, 

Date: October 10, 2006 r... Kaiie, jr./ 

Robert'LrHailsj'r" 
Registration No. 39,702 

KErNYON & KENYON LLP 
1500 K Street, N.W. 
Washington, D.C 20005 
Ph.; (202) 220-4200 
Fax.: (202)220-4201 
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